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Abstract. In this paper we define a new condition number adapted to directionally uniform 
perturbations. The definitions and theorems can be appUed to a large class of problems. We show 
the relation with the classical condition number, and study some interesting examples. 
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CN I Let X and Y be two real (or complex) Riemannian manifolds of real dimensions m and n 

^ [ [m > n) respectively associated to some computational problem, where X is the space of inputs 

OO '. and Y is the space of outputs. Let V d X x Y he the solution variety, i.e. the subset of pairs 

'. (x, y) such that y is an output corresponding to the input x. Let ni : V ^ X and -K2 : V Y he 



1. Introduction 



the canonical projections, and S C be the set of critical points of the projection tti. 

In case dim = dimX, for each (x, y) G V \ S there is a differentiable function locally defined 
from some neighborhoods and Uy oi x & X and y eY respectively, namely 

G := 7r2 o Tv^^lu^ : Uy. 

Let us denote by {■,-)x and {■,-)y the Riemannian (or Hermitian) inner product in the tangent 
spaces T^X and TyY at x and y respectively. The derivative DG{x) : T^X TyY is called the 
condition matrix at {x,y). The classical condition number at {x,y) E V \ is defined as 

(1) hi{x,y) := max ||DG(a:)i;||y. 

l|i||^=i 

This number is an upper-bound -to first-order approximation- of the worst-case sensitivity of 
the output error with respect to small perturbations of the input. It plays an important role to 
understand the behavior of algorithms and, as a consequence, appears in the usual bounds of the 
running time of execution. There exist an extensive literature about the role of the condition 
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number in numerical analysis and complexity of algorithms, see for example [13] and references 
therein. 

In many practical situations, there exist a discrepancy between theoretical analysis and ob- 
served performance of an algorithm. There exist several approaches that attempt to rectify this 
discrepancy. Among them we find average-case analysis (see [Zl [lO]) and smooth analysis (see 
[m m [15]). For a comprehensive review on this subject with historical notes see [5]. 

In this paper, averaging is performed in a different form. In many problems, the space of 
inputs has a much larger dimension than the one of the space of outputs (m n). Then, it is 
natural to assume that infinitesimal perturbations of the input will produce drastic changes in the 
output, only when they are performed in a few directions. Then, a possibly different approach to 
analyze complexity of algorithms is to replace "worst direction" by a certain mean over all possible 
directions. This alternative was already suggested and studied in [H] in the case of linear system 
solving Ax = b, and more generally, in |12| in the case of matrix perturbation theory where the 
first-order perturbation expansion is assumed to be random. 

In this paper we extend this approach to a large class of computational problems, restricting 
ourselves to the case of directionally uniform perturbations. 

Generalizing the concept introduced in [14] and [12], we define the pth-average condition number 
at (x, y) as 

i/p 



(2) naJ'\x,y) 



1 



Lvol(^," 



m—1'' 



\\DG{x)x\\ldS^~\x 



(P=l,2, 



where vol(S'^~^) = ^^j^-^ is the measure of the unit sphere S^"-^ in T^X, and dS^~^ is the 
induced volume element. We will be mostly interested in the case p = 2, which we simply write 
Kav and call it average condition number. 

Before the statement of the main theorem, we define the Frobenius condition number as 



KF{x,y) := \\DG{x)\\f= \/ c^f + ■ ■ ■ + 
where || ■ \\f is the Frobenius norm and ai, . . . , cr„ are the singular values of the condition matrix. 

Theorem 1. 



i^av^'''\x, y) — 



r(^) 

where \\ ■ \\ is the Euclidean norm in and ?7(ji,...,(t„ is a centered Gaussian vector in M" with 
diagonal covariance matrix Diag^af, . . . , cr^). 
In particular, 

(3) Kav{x,y) = 

This result is most interesting when n <^m, for in that case 

t^avix, y) < ■ k{x, y) < k(x, y), 
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as K,F{x,y) < \fn- K,{x,y). Thus, in these cases one may expect much better stability properties 
than those predicted by classical condition numbers. 

In numerical analysis, many authors are interested in relative errors. Thus, in the case {X, \\ ■ \\x) 
and (y, II ■ II y) are normed vector spaces, instead of consider the (absolute) condition number ([T]), 
one could take the relative condition number defined as 



and the relative Frohenius condition number as 



\x\\x 

\y\\Y 



K{x,y), 



m\x 
IvWy 



(for simplicity we drop the y in the argument). 
In the same way, we define the relative pth-average condition number as 



(4) 



\x\\x ^ 

WvIIy' 



^aJ^\x,y), (p = l,2,...). 



For the case p = 2 we simply write K^eiav ^tnd call it relative average condition number. 

r/ieorem [I] remains true if one change the (absolute) condition number by the relative condition 
number. In particular, 

KrelF{x,y) 



m 



2. Componentwise Analysis 
In the case F = M*^ we define the kth- componentwise condition number at (x, y) E V as: 



(5) 



hi{x,y,k) := max \{DG{x)x)k\ {k = l,...,n), 

llxlp =1 



where | ■ | is the absolute value and Wk indicates the fcth-component of the vector w eW^. 
Following [H] for the linear case, we define the average componentwise condition number as 



(6) Ka.^P\x,y,k) :-- 
Then we have: 



vol(5™-i) 7,e5r- 



\{DG{x)x)kf dS^-\x 



i/p 



(p=l,2,...). 



Proposition 1. 



f^av^^^i^^ yi k) 



In particular, 



r(^) 



f^avi^: y-, k) 



r(f) _^fP+i 



HF{x,y,k) 



m 



i/p 



KF{x,y,k). 
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Proof. Observe that Kav^^\x,y, k) is the pth-average condition number for the problem of finding 
the fcth-component of G = (Gi, . . . , Gn). T/teorem C] apphed to Gk yields 



Hi 



r(f) 
r(^) 



o-i I 



\p\l/p 



where ai = \\DGk{x)\\ = n{x,y,k). Then, 

where r/i is a standard normal in M. Finally. 
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K{x,y,k) - Ed?]] 



\p\i/p 



and the proposition follows. 



ffe-"'/^ dp 



2 ^E^„,p + 1. 



27r 



2— r 



□ 



3. Examples 

In this section we will compute the average condition number for different problems: systems 
of linear equations, eigenvalue and eigenvector problems, finding kernels of linear transformations 
and solving polynomial systems of equations. The first two have been computed in [12] and are 
an easy consequence of TheoremUl ami the usual condition number. 

3.1. Systems of Linear Equations. We consider the problem of solving the system of linear 
equations Ay = b, where A e ^„(M) the space of nxn matrices with the Frobenius inner product, 
i.e. {A, B)f = trace(5*A) (5* is the transpose of 5), and h G W. 

If we assume that h is fixed, then the input space X = ^„(M) with the Frobenius inner product, 
F = M*^ with the Euclidean inner product, and S equals the subset of non-invertible matrices. 
Then the map G : ^„(M) \ S — > M" is globally defined and differentiable, namely 

GiA) = A-'b i=y). 

By implicit differentiation, 

(7) DGiA){A) = -A~'Ay. 

Is easy to see from ((71) that K{A,y) = \\A~^\\ ■ \\y\\ and KF{A,y) = \\A~^\\f ■ \\y\\. Then from 
T/ieorem H we get 

(8) .„.(^,,, = M:^^<«('4.!') 



n 

A similar result was proved in [T2] . 

For the general case, we have X = ^„(M) x M" with the Frobenius inner product in ^„(R) 
and the Euclidean inner product in M". Then, G : ^„(M) \ S x M" satisfies G{A, h) = A-^b. 



Is easy to see that K{{A,b),y) = \\A ^\\ ■ a/1 + ||yP and KF{{A,b),y) = \\A ^\\f ■ \J\ + |||/P- 
Again from T/zeorem [I] we get 

V "^i + n \jn + 1 
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For the A;th-componentwise condition number, we have that 



KaviiA,b),y,k) 



-1 1/p 



■ r 



p + 1 



K{{A,b),y). 



A similar result was proved in [T4] . 

In [7], it is proved that the expected value of the relative condition number Krei{A) = \\A\\ ■ \\A'^\ 
of a random matrix A whose elements are i.i.d standard normal, satisfies: 

E{\og Krel{A)) = logra + C + o(l), 

as m ^ oo, where c ~ 1.537. If we consider the relative average condition number defined in ([4]) 
we get from ([8]) 

1 



E(logKrelav{A)) = - logn + C + o(l) 



as m OO. 



3.2. Eigenvalue and Eigenvector Problem. Let X = ^„(C) be the space of n x n complex 
matrices with the Frobenius Hermitian inner product, Y = P(C'^+^) x C and V = {{A,v,X) : 
Av = Xv}. Then for {A,v, A) G \ S the condition matrices DGi and DG2 associated with the 
eigenvector and eigenvalue problem are 

(n, Av) 
{u,v) 

where u is some left eigenvector associated with A, i.e. u*A = Xu* (see [Bez IV]). The associated 
condition numbers are: 



DGi{A)A = {iT^±{XI - A)l±y^ (vr^xit;) and DG2{A)A 



(9) Ki{A,v) = \\{tt^±{XI - A)l±) ^\\ and K2{A, X) = 

I \^) ^7 I 

From our TheoremUl we get the respective average condition numbers: 

Kavl{A,v) = - ||(7r„±(A/ - < -^Ki{A,v), 

Kav2{A>^) = -l^2{A A). 

n 

A similar result for Kav2{A, A) was proved in [T2] . 

3.3. Finding Kernels of Linear Transformations. Let ^k,piC) be the linear space of k x p 
complex matrices with the Frobenius Hermitian inner product, i.e. {A,B)f = trace(-B*A) {B* is 
the adjoint of B), and TZr C ^/j p(C) the subset of matrices of rank r. We consider the problem of 
solving the system of linear equations Ax = 0. For this purpose, we introduce the Grassmannian 
Gp/ of complex subspaces of dimension d in C^, where £ = dim(ker A) = p — r . 

The input space X = 7^^ is a smooth submanifold of ^k,p{G) of complex dimension {k+p)r — r"^ 
(see [6]). Thus, it has a natural Hermitian structure induced by the Frobenius metric on ^k,p{ 

In what follows, we identify Gp^e with the quotient E>pj/Ui of the Stiefel manifold 

:= {M G ^p,e{C) : M*M = 1} 
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by the unitary group lA^ C ^^(C), which acts on the right of Sp,^ in the natural way (see [6]). 
Then, the complex dimension of the output space Y = Gp^ is {p — r)r. 

We will use the same symbol to represent an element of Sp/ and its class in Gpj. The manifold 
Sp^i has a canonical Hermitian structure induced by the Frobenius norm in ^e{C). On the 
other hand, Ui is a Lie group of isometrics acting on E>p^g. Therefore, Gp/ is a Homogeneous 
space (see [8]), with a natural Riemannian structure that makes the projection Sp,^ — Gp^e a 
Riemannian submersion. The orbit of M G Sp/ under the action of the unitary group Ue, namely, 
oi{M) = {MU : U G Ui}, defines a smooth submanifold of Sp^i. In this form we can define a local 
chart of a small neighborhood of M G Gp/ from the affine spaces 

M + TMOe{M)^ 

where TmOi^M)^ is the orthogonal complement of TmO^(M) in Tm^p/- Implicit differentiation in 
local coordinates of the input-output map G at the point G{A) = M G Sp,^ yields 

(10) DG{A){A) = -A^Am, 

where A G T/R-r, A'^ is the Moore-Penrose inverse of A and A^ AM G Tmo^(M)-'-. 

One way to compute the singular values of the condition matrix described in ( ITOl ). is to take 
an orthonormal basis in ^fc p(C) which diagonalizes A. From the singular value decomposition, 
there exists orthonormal basis . . . of C''', and {vi, . . . ,Vp} of C^, such that Avi = diUi 
for i = 1, . . . ,r, and Avi = for i = r + 1, . . . Thus, {uiV* : i = 1, . . . , k; j = 1, . . . , p} is an 
orthonormal basis of ^k,p{G) which diagonalizes A. In this basis the tangent space T/R-r is the 
orthogonal complement of the subspace generated by {uiV* : i = r + 1, . . . ,k; j = r + 1, . . . ,p}. 
From where we conclude that 

KiAM) = \\DGiA)\\ = pt||, KpiA,M) = Vp^- PIf. 

From our TheoremU\ 



yj{k + p-r)r \l[k + p-r)r 



In [T], it is proved that 

¥.{\0gKrel{A) : AeUr) < log 



k + p — r 
k + p-2r + l 



2.6, 



where the expected value is computed with respect to the normalized naturally induced measure 
in TZj.. Our T/ieorem [1] immediately yields a bound for the average relative condition number, 
namely, 



E(\0gKrelav{A) : A G 7^,) < ^ log 



{k + p — r)r 



{k + p — 2r + l)'^p{p — r) 



2.6. 
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3.4. Finding Roots Problem I: Univariate Polynomials. We start with the case of one 
polynomial in one complex variable. Let X = Vd = {f '■ f{z) = Ylt=ofi^^^ /« ^ Identifying 
Vd with C^^^ we can define two standard inner products in the space Vd- 

- Weyl inner product: 

(11) {f,9)w-=J2f^M'^) '5 

- Canonical Hermitian inner product: 

d 

(12) {f,9)cd+^ ■=^fi9i- 

The solution variety is given by = {(/, z) : f{z) = 0}. Thus, by implicit differentiation 

We denote by kw and n^d+i the condition numbers with respect to the Weyl and Euclidean inner 
product. The reader may check that 



,,,, (1 + Kir^' , ,,,, yEtoici^' 

t^w{fX) = — ^^T^^^i — and Kcd+i{fX) = — ^j^^ — , 
(for a proof see [3], p. 228 ). From TheoremUi we get: 

V2("+1) V2(d + 1) 

3.5. Finding Roots Problem II: Systems of Polynomial Equations. We now study the 
case of complex homogeneous polynomial systems. Let TC^d) the space of systems / : C""*^^ — ^ C", 
/ = (/i, . . . , fn) where each fi is a homogenous polynomial of degree di. We consider H(d) with 
the homogeneous analogous of the Weyl structure defined above (see Chapter 12 of [3] for details). 

Let X = P(H(d)) and Y = P(C"+^) and V = {(/,() : /(C) = 0}. We denote by = 
Sr=i ("'^n ") ~ ■'■ complex dimension of X. We may think of 2N as the size of the input. 
Then, 

DG{f)if) = -{DfiOc^y'fiC), 

and the condition number is 

Kwifx) = \\{Df{Oc-y'\ ^ 

where some norm 1 affine representatives of / and ( have been chosen (cf.[3]). Associated with 
this quantity, we consider 



(13) ^^(/)^=w^ E '^^(/'O 



V 

{0/(0=0} 



where V = di ■ ■ ■ dn is the number of projective solutions of a generic system. 

The expected value of Hyy{f) is an essential ingredient in the complexity analysis of path- 
following methods (cf. [9], [2]). In [2] the authors proved that 

(14) Ef [t^wif))'] < SnN, 
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where / is chosen at random with the Weyl distribution. 

The relation between complexity theory and Kav is not clear yet. However, it is interesting to 
study the expected value of the Kau-analogous of equation ( fT4l ). namely 



{0/(0=0} 



From our T/ieoremQ we get, 



Note that the last bound depends on the number of unknowns n, and not on the size of the input 
n < iV. 



4. Proof of the main Theorem 

In the case of complex manifolds, the condition matrix turns to be an n x n complex matrix. 
In what follows, we identify it with the associated 2n x 2n real matrix. We center our attention 
in the real case. 

The main theorem follows immediately from Lemma [I] and Proposition [H below. 



Lemma 1. Let rj he a Gaussian standard random vector in 



Then 



r(f) 



mDG{xnn 



p\]i/p 



■11 is the Euclidean norm in 



where E is the expectation operator and \\ 

Proof. Let / : ^ M be the continuous function given by 

f{v) = \\DG{x)v\\. 

Then 



Integrating in polar coordinates we get that 



f{v) 



1/p 



(15) 
where 



n\\DG{xH\^ 



(27r)™/2 



fP dS' 



+ 00 



Making the change of variable u = /2 Mve obtain 



therefore 
(16) 



'm+p— 1 



Then joining together (fTSl) and (fTBl) we obtain the result. 



□ 
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Proposition 2. 

Ei\\DG{x)vr) = mv.u...,<^jn 

where r7cri,...,o-„ is a centered Gaussian vector in vjith diagonal covariance matrix Diag^af, . . . , a^) . 

Proof. Let DG{x) = UDV be a singular value decomposition of DG{x). By the invariance of 
the Gaussian distribution under the action of the orthogonal group in X, Vrj is again a standard 
Gaussian random vector. Then, 

E(||DG(x)r/r)=E(||f/Z}ryr), 

and by the invariance under the action of the orthogonal group of the Euclidean norm, we get 

E(pG(x)r/r)=E(pr/r). 

Finally Drj is a centered Gaussian vector in M" with covariance matrix Diag{af, . . . , cr^), and the 
proposition follows. For the case p = 2 

K,,{x, y) = [E [alril + ... + alril)] 

where 771, . . . ,?7„ are i.i.d. standard normal in M. Then 

□ 
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